Tornado: A Practical And Efficient Heterogeneous Programming Framework For Managed Languages

Hardware accelerators o‚er developers the ability to improve the performance and energy eciency of their applications. However, a key barrier that is preventing their widespread adoption are the shortcomings of existing tools for heterogeneous programming — they are not designed to handle the complexity of deploying an application into an environment where the availability and type of hardware accelerators are unknown or cater for developers that use managed languages to process “Big Data”. Œis paper describes our experiences creating Tornado: a practical and ecient heterogeneous programming framework for managed languages. Œe novel aspect of Tornado is that it turns the programming of heterogeneous systems from an activity predominantly based on a priori knowledge into one based on a posteriori knowledge. Alternatively put, it simply means developers do not need to overcomplicate their code by catering for all possible eventualities. Instead, Tornado provides the ability to specialize each application for a speci€c system in situ which avoids the need for it to be pre-con€gured by the developer. To enable this, Tornado employs a sophisticated runtime system that can dynamically con€gure all aspects of the application — from selecting which parallelization scheme to apply to specifying which accelerators to use. By using this ability, the end-user, and not the developer, can transparently make use of any available multior many-core processor and accelerators. To showcase the impact of Tornado, we implement a real-world computer vision application and deploy it across nine accelerators without having to modify the source code or even explicitly re-compile the application. Using dynamic con€guration, we show that our implementation can achieve up to 124 frames per second (FPS) up to 166× speedup over the reference implementation. Finally, our implementation is always within 21% of a hand-wriŠen OpenCL version but avoids much of the programming tedium.